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Introduction 


College and career readiness depends on students acquiring a complex set of knowledge and skills. 
Greater workforce demand for employees with advanced abilities has resulted in a growing emphasis 
on more rigorous academic standards to ensure that students are prepared for success, including 
at least some postsecondary education or training. However, although these standards delineate 
content knowledge and academic skills in English language arts and mathematics, students must 
master a broader set of competencies to ensure that they are truly on track for college and a career. 
In this annotated bibliography, we refer to that broad set of competencies as "lifelong learning 
skills.” These are the essential behaviors and abilities that contribute to a learner’s intellectual 
and emotional maturity, professionalism, and good citizenship. We define these competencies 
in the College and Career Readiness and Success (CCRS) Organizer. 

The College and Career Readiness and Success Organizer is a graphic (see Figure 1) that provides 
a consolidated overview of the many institutional and individual elements that influence a learner’s 
ability to succeed in college and careers. 

Figure 1. College and Career Readiness and Success Organizer 
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Included in the organizer are the lifelong learning skills or competencies often referred to as 
“21st century skills,” non-cognitive factors, employability skills, hard-to-measure skills, or skills for 
deeper learning. Throughout this annotated bibliography, we will use the term “lifelong learning 
skills” to refer to the set of competencies discussed across the sources represented; however, 
in each annotation we will use the terminology favored by the author of each piece. 

The collective contribution of several academic disciplines to the study of lifelong learning skills 
yields the most comprehensive and compelling case for attending to these foundations for 
learning and working to promote college and career readiness. Looking across disciplines allows 
us to explore skills long thought to be important both inside and outside of the classroom — as 
well as throughout a person’s lifespan — with the unique insight granted by the distinct lenses 
through which each discipline has studied them. Research in education, psychology, and labor 
economics has identified the set of competencies that makes up lifelong learning skills as 
positively associated with academic achievement and other aptitudes that are important 
precursors to, and elements of, postsecondary and career success. 

Although the field has yet to reach consensus on the terminology used across disciplines to refer 
to lifelong learning skills, as educators, administrators, and policymakers strive to meet college 
and career readiness and success goals, it is imperative that these skills are well understood. In 
order to promote the use of consistent terminology, we have focused on the four lifelong learning 
skills most frequently identified in research: social and emotional learning (SEL), higher-order 
thinking skills, application of knowledge in cross-disciplinary contexts, and academic success 
and employability skills. We have mapped these components of the lifelong learning skills framework 
onto the widely used, three-part categorization of competency types: cognitive, intrapersonal, and 
intrapersonal (see, for example, National Research Council, 2012). This organization of lifelong 
learning skills is the most compatible with the majority of the sources we reviewed. 

The sources contained within this annotated bibliography can help inform state efforts to define 
the competencies that students need to be able to demonstrate, determine how schools and 
districts can ensure that students master these competencies, and measure school and student 
progress toward college and career readiness and success goals. This annotated bibliography 
seeks to provide a point of reference for achieving these objectives. It catalogs resources that 
explain what lifelong learning skills are, how they impact and relate to college and career 
readiness and success outcomes, and how they can be taught and measured in the classroom, 
across the prekindergarten to workforce (PK-20W) spectrum, so that users can make informed 
decisions about which topics to explore in further detail. It should be noted, however, that although 
many of these resources demonstrate a direct link to college and career readiness, most focus on 
academic achievement more broadly and were conducted with learners at various stages of the 
PK-20W spectrum. More research is needed to ensure that these important competencies are 
translated into validated indicators of college and career readiness outcomes, such as obtaining 
a diploma and enrolling in and graduating from postsecondary education or training. 

The selected resources provided here, and their accompanying annotations, represent a 
comprehensive overview of lifelong learning skills research. Based on the types of reports and 
studies available in the field, the resources are organized into four sections: Major Reports, Lifelong 
Learning Skills in Depth, Impact on Career Readiness, and Measures. The first section includes a 
series of overview reports that provide frameworks for or relationships between components of the 
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entire lifelong learning skills landscape. The second section presents studies that examine individual 
or smaller subsets of lifelong learning skills. Finally, in the last two sections, we highlight research 
that examines two particular points of interest for policymakers and educators: career readiness 
and the assessment of lifelong learning skills. The organization of the guide provides a detailed 
and issue-specific review of lifelong learning skills, which allows the reader to approach the 
document according to his/her particular interests or needs . 1 


Methodology and Limitations 

The search process for the sources contained within this annotated bibliography began with the 
existing foundation provided by the works cited in the Major Reports section presented here. The 
majority of these sources had already been identified by the CCRS Center as part of an ongoing 
effort to maintain a research database of high-quality literature in the field of college and career 
readiness and success. Additional sources for the Major Reports section were added based on 
the recommendation of other experts in the field. 

After completing this process, we expanded on that resource base by searching for additional 
resources using search terms that corresponded with the terminologies, taxonomies, and 
frameworks developed in the Major Reports section . 2 The search engines used were Google 
Scholar, PsycINFO, Academic Search Premier, JSTOR, and ERIC. Sources were chosen for 
review based on recent publication dates and the number of times they had been cited in 
other reports. 

During our search, we found that the research base for interpersonal skills was particularly 
limited. This gap in available research corresponds with the findings of several authors of the 
major reports. Few studies examine interpersonal skills in isolation from intrapersonal skills, 
and those that do rarely study specific, individual interpersonal aptitudes, such as leadership, 
communication, and collaboration. Instead, the majority of the research on components of 
interpersonal skills is included in studies that investigate the connection between SEL and 
academic success. We have annotated the sources that feature SEL, chosen for this cluster 
of skills, to emphasize the interpersonal aspects of their findings. In addition, a number of the 
economics studies included in the Impact on Career Readiness section of this document explore 
the importance of these interpersonal skills for predicting career success. Despite these related 
areas of study, more research is needed to investigate specific components of interpersonal 
skills that may impact college and career readiness and success. 


1 Where possible, we have provided a publicly available direct URL link to each source cited. In addition, the appendix 
contains a chart organizing the sources according to topic, type of report, and potential utility. 

2 Search terms included the following: “21st century skills,” hard-to-measure skills, deeper learning, lifelong learning 
skills, non-cognitive skills, non-cognitive factors, cognitive skills, interpersonal skills, intrapersonal skills, problem 
solving, complex problem solving, creativity, critical thinking, cognitive strategies, meta-cognition, self-regulation, 
motivation, mindsets, personality, communication skills, leadership skills, collaborative problem solving, knowledge 
transfer, measures, assessment, academic achievement, social and emotional skills, SEL, career achievement, career 
success, career readiness, college readiness, teaching, learning, performance, K-12, elementary school, middle school, 
college, and postsecondary. 


Lifelong Learning Skills for College and Career Readiness 


The search process for this resource was designed to be as comprehensive as possible to gather 
information on how lifelong learning skills contribute to college and career readiness and success; 
however, the research is rich and diverse and any search process will necessarily include limitations. 
As such, the included annotations represent an effort to provide a range of available resources, 
and the exclusion of any given source is not intended to imply its lack of value to the field. Similarly, 
the inclusion of a source does not represent any evaluation of its methodologies or findings. 


Major Reports 

This section provides a set of comprehensive, guiding research reviews that lay the conceptual 
foundation for understanding what lifelong learning skills are and why they matter in education. 
Each document explains lifelong learning skills with reference to a specific purpose and audience. 
Although some of these documents primarily address the concerns of researchers (e.g., taxonomy 
and measurement), others are more explicitly tailored to the concerns of educators, administrators, 
and policymakers (e.g., pedagogy, assessment, and outcomes). The annotation for each source 
identifies the focus of the document and highlights its key findings. Each source attempts to 
answer one or more of the following guiding questions: 

1. What are lifelong learning skills? 

2. How can lifelong learning skills be accurately identified and assessed? 

3. How can lifelong learning skills be fostered and/or taught by practitioners, 
including administrators and teachers? 

National Research Council. (2014). Education for life and work: Guide for practitioners. 

Committee on Defining Deeper Learning and 21st century skills, J. W. Pellegrino and M. L. 
Hilton, Editors. Board on Testing and Assessment and Board on Science Education, Division of 
Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. 

This guide to “21st century skills” is intended for state-, district-, and school-level administrators. 
It condenses the research presented in the report of the same name (National Research 
Council, 2012), focusing on teaching practice and the intersection of "21st century skills” with 
the Common Core State Standards. It describes the relationship between “21st century skills” 
and deeper learning; explores best practices for integrating these skills within the context of 
teaching English language arts, mathematics, and science; and provides guidance on how to 
develop and support a schoolwide culture that is founded upon teaching and learning skills 
for lifelong learning. 

Soland, J., Hamilton, L. S., & Stecher, B. M. (2013). Measuring 21st century competencies: 
Guidance for educators. New York, NY: Asia Society. 

This report is explicitly intended for teachers, school leaders, and administrators. Its primary 
focus is to introduce the reader to the assessments currently available to measure specific 
“21st century competencies” and implications for implementation in K-12 education settings. 
The authors present a framework for selecting assessments, including considerations for school 
leaders and administrators (such as cost, logistics, technical capacity, and systemwide impact) 
and for teachers (such as instructional practice and training). The authors provide a review 
of test formats (e.g., multiple-choice versus performance assessments) and the practical 
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and technical considerations associated with each of them, along with 10 case studies of 
“cutting-edge” assessments of 21st century competencies. 

National Research Council. (2012). Education for life and work: Developing transferable 
knowledge and skills in the 21st century. Committee on Defining Deeper Learning and 21st 
century skills, J. W. Pellegrino and M. L. Hilton, Editors. Board on Testing and Assessment 
and Board on Science Education, Division of Behavioral and Social Sciences and Education. 
Washington, DC: The National Academies Press. 

This report examines the research on “21st century skills,” spanning the fields of psychology, 
education, and economics, where the majority of this research has been conducted. It begins 
with a systematic, research-based definition and taxonomy of “21st century skills,” exploring 
their relationship with deeper learning. The authors then review the research for each cluster 
of skills (i.e. , cognitive, intrapersonal, interpersonal) with respect to its impact on academic 
performance within the subjects of English language arts, mathematics and science, and career 
performance. They review teaching and intervention methods to develop these skills and abilities, 
assessments, and approaches to developing school cultures and curricula that support 
education based on deeper learning and the development of "21st century skills.” 

Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & 
Beechum, N. 0. (2012). Teaching adolescents to become learners. The role of noncognitive 
factors in shaping school performance: A critical literature review. Chicago, IL: University of 
Chicago Consortium on Chicago School Research. 

This report introduces a framework for understanding how non-cognitive factors interact 
to influence academic achievement (see Figure 2 below). 

Figure 2. How Five Non-Cognitive Factors Affect Academic Performance 



Lifelong Learning Skills for College and Career Readiness 












For each factor, the authors provide an overview of the latest research, and — where possible — 
examples of real-world implementation of strategies designed to promote that factor. The authors 
highlight mindsets and motivation as particularly promising leverage points for educators interested 
in improving students’ academic achievement and college and career readiness. The authors caution 
that although it may be tempting to provide explicit instruction and directly assess non-cognitive 
skills, this may not be an effective strategy for developing them. There are, to date, limited data to 
suggest that these skills are malleable and little evidence that teaching them explicitly, in isolation, 
can have an impact. The authors’ recommendations call for more research to explore the issues of 
malleability and teachability, and they provide actionable strategies for the promotion of non-cognitive 
factors to improve student performance. 

Saavedra, A. R., & Opfer, V. D. (2012). Teaching and learning 21st century skills: Lessons from the 
learning sciences. Global Cities Education Network. 

This report reviews nine "21st century skills” teaching objectives (see Figure 3 below) designed 
to address student cognitive, interpersonal, and intrapersonal domains. The authors provide 
examples of teaching strategies designed to address each objective, and conclude with a brief 
discussion of student assessment challenges and the need for teacher professional development 
to support the shift to teaching “21st century skills.” 

Figure 3. Nine 21st Century Teaching Objectives 


How to Teach “21st Century Skills”: Nine Lessons From the Learning Sciences 

1. Make It Relevant 

2. Teach Through the Disciplines 

3. Simultaneously Develop Lower- and Higher-Order Thinking Skills 

4. Encourage Transfer of Learning 

5. Teach Students to Learn to Learn 

6. Address Misunderstandings Directly 

7. Understand That Teamwork Is an Outcome and Promotes Learning 

8. Exploit Technology to Support Learning 

9. Foster Students' Creativity 

National Research Council. (2011). Assessing 21st century skills: Summary of a workshop. J. A. 

Koenig, Rapporteur. Committee on the Assessment of 21st Century Skills. Board on Testing 
and Assessment, Division of Behavioral and Social Sciences and Education. Washington, DC: 
The National Academies Press. 
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The focus of this report is the need for, and feasibility of, assessing “21st century skills." 

The authors examine the research base for the measurement of cognitive, interpersonal, 
and intrapersonal skills with a view toward using assessments for accountability purposes 
and improving instruction. A broad consensus emerges from the research: Cognitive skills 
are largely subject-specific and cannot be taught or tested in a subject-neutral manner — 
that is, they operate idiosyncratically within the context of an academic subject or discipline. 
Inter- and intrapersonal skills, on the other hand, are considered by the authors to be broadly 
applicable. They can be explicitly taught and monitored with formative assessments. The 
authors then provide a discussion of the implications of their findings for educators, school 
leaders, policymakers, and researchers. 


Lifelong Learning Skills in Depth 

This section organizes the research on lifelong learning skills according to the three primary 
groupings of competencies most commonly found in the research: cognitive competencies, 
intrapersonal competencies, and interpersonal competencies (see, for example, National Research 
Council, 2012; Soland, Hamilton, & Stecher, 2013). Cognitive competencies (also commonly referred 
to as higher-order thinking skills) include creativity, critical thinking, and problem solving; intrapersonal 
competencies include self-regulation and motivation; and interpersonal competencies include social 
and emotional competencies. The articles examine individual or subsets of lifelong learning skills 
and their potential impact on academic achievement and college and career readiness. 


COGNITIVE COMPETENCIES: CREATIVITY, CRITICAL THINKING, 

AND PROBLEM SOLVING 

Greiff, S., & Funke, J. (2013). Complex problem solving in educational contexts — Something 
beyond g: Concept, assessment, measurement invariance, and construct validity. Journal of 
Educational Psychology, 105(2), 364-379. 

This study provides background on the nature of complex problem solving (CPS). The authors 
describe how CPS is distinct from general intelligence (g) and how CPS strategies are used to 
solve problems across subjects. They describe two main processes involved in CPS: knowledge 
acquisition and knowledge application. In contexts where prior knowledge is peripheral to solving 
the problem, a student’s ability to use CPS strategies (e.g., heuristics), which are applicable 
across subjects, can be observed. The authors suggest that more research is needed to 
determine the best means of promoting CPS in the classroom; however, they cite examples 
of teaching strategies that abstract the principles of CPS — such as direct instruction in 
the scientific method — which had the effect of improving CPS and knowledge transfer 
across subjects. 
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Kuncel, N. R., & Hezlett, S. A. (2010). Fact and fiction in cognitive ability testing for admissions 
and hiring decisions. Current Directions in Psychological Science, 19(6), 339-345. 

This study provides a review of the research on cognitive ability testing used for admissions and 
hiring decisions. The authors come to four conclusions about the relationship between general 
cognitive ability and academic and career performance: 

1. There is a relationship between performance on cognitive ability tests and 
creativity/leadership. 

2. Tests are accurate across race and sex. 

3. Tests are predictive of later academic and career performance, independent from 
socioeconomic status. 

4. Other characteristics, such as personality (specifically, conscientiousness, which is related 
to self-regulation and perseverance), are important factors for predicting performance. 

Klauer, K.,& Phye, G. (2008). Inductive reasoning: A training approach. Review of Educational 
Research, 78(1), 85-123. 

This study reviews three programs designed to train children and adolescents (ages 5-16) 
in inductive reasoning. Each program consists of 10 standardized lesson plans and test 
questions to assess progress. The evidence suggests that it is possible to directly train 
children of a wide age range and cognitive ability to effectively practice inductive reasoning, 
using each of the three training programs developed (one for ages 5-8, one for ages 11-13, 
and another for ages 14-16), and that such training can produce a moderate effect on 
measures of academic performance. 

Mayer, R. E., & Wittrock, M. C. (2006). Problem solving. In P A. Alexander & P H. Winne (Eds.), 
Handbook of educational psychology (pp. 287-304). Mahwah, NJ: Erlbaum. 

This chapter, within the Handbook of Educational Psychology, provides a comprehensive review 
of problem solving in education. It includes a set of seven best-practice methods (see Figure 4 
on next page) for teaching problem-solving skills and establishes three guiding principles, for 
those in education and research to follow, when considering problem solving as a primary goal 
of instruction: 

1. Problem-solving skills are learned best within domain-specific contexts. 

2. Problem-solving skills are restricted in their range of applicability across subjects. 

3. Whether dealing in factual and procedural knowledge, or conceptual and strategic 
knowledge, the context of guided problem-solving tasks is essential for student learning. 
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Figure 4. Seven Ways to Promote Problem-Solving Transfer 


Instructional Method 

Example 

Load-reducing methods 

Automaticity, constraint removal 

Structure-based methods 

Concrete manipulatives 

Schema-based methods 

Advance organizers, pre-training, cueing 

Generative methods 

Elaboration, note-taking, self-explanation, questioning 

Guided discovery methods 

Guided discovery 

Modeling methods 

Worked examples, apprenticeship 

Teaching thinking skills 

General courses, specific strategies 


Scott, G., Leritz, L. E., & Mumford, M. D. (2004). The effectiveness of creativity training: A 
quantitative review. Creativity Research Journal, 16(4), 361-388. 

This report provides a meta-analysis of 70 studies of creativity training programs. The authors 
find that there is compelling evidence for effective creativity training programs that have positive 
effects on student problem solving and performance on tasks requiring creative thinking. These 
effects were evident across content areas, settings, and student groups. The most effective 
programs emphasize cognitive strategies designed to evaluate and define problems. Example 
strategies include convergent thinking, 3 problem identification, and constraint identification. 4 The 
least effective programs emphasized “unconstrained exploration,” such as expressive activities, 
elaboration, and use of imagery. The authors also found that programs that used a model-based 
approach for creativity training and focused on practicing the strategies from that model within 
specific domains (subjects) were the most successful. They highlight two programs, the Purdue 
Creative Thinking Program and the Creative Problem-Solving Program, as being exemplary models. 

Salomon, G., & Perkins, D. N. (1989). Rocky roads to transfer: Rethinking mechanisms of a 
neglected phenomenon Educational Psychologist, 24, 113-142. 

Transfer of knowledge refers to when learning in one domain or context (e.g., mathematics) is 
successfully applied for a different purpose in another domain or context (e.g., biology). This 
study investigates the conditions under which transfer of knowledge occurs. The authors argue 
that there are two kinds of transfer: low-road and high-road. They argue that low-road transfer of 
simple skills (e.g., tying knots) occurs with repeated practice and is triggered automatically by a 
context that requires it. On the other hand, high-road transfer of complex skills or concepts 


3 A process through which learners access prior knowledge to choose the best answer choice and eliminate others 

4 Designed to help learners evaluate and delineate a problem 
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(e.g., graphically modeling problems in mathematics and science) occurs only with “intentional, 
mindful abstraction of something from one context” to another. This distinction carries with 
it implications for pedagogy when attempting to foster transfer of knowledge. Specifically, it 
suggests that for some simple skills, repeated practice is sufficient for students to generalize 
mastery from one content area to another, whereas for more complex skills, explicit instruction 
in the process of mindful abstraction is necessary. 

Gick, M. L, & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive 
Psychology, 15(1), 1-38. 

This study demonstrates a method for teaching students problem-solving strategies (e.g., divide 
and conquer) that can be applied across parallel problems. First, students are presented with 
two problem-solution “stories” and are then asked to summarize the stories, provide a verbal 
description of the similarities between the two stories, or create a diagram to represent the 
stories. After this application, students were able to successfully apply the strategy featured 
in the story to new, analogous problems. The findings suggest that this method may be an 
effective way to teach problem-solving strategies that can be applied across different 
problem contexts. 


INTRAPERSONAL COMPETENCIES: SELF-REGULATION 
AND PERSONALITY; MINDSETS AND MOTIVATION 


Self-Regulation 

Duckworth, A. L., Grant, H., Loew, B., Oettingen, G., & Gollwitzer, P M. (2011). Self- regulation 
strategies improve self-discipline in adolescents: Benefits of mental contrasting and 
implementation intentions. Educational Psychology, 31(1), 17-26. 

This study details the results of a brief self-regulation training intervention with 66 high school 
students preparing for a high-stakes exam. The strategies taught were mental contrasting and 
implementation intentions, which are evidence-based strategies that foster active thinking and 
planning about future goals. Students who participated in the intervention demonstrated a 
significantly greater commitment to test preparation, as evidenced by the number of practice 
problems attempted. The authors' findings suggest that directly teaching a combination of 
these two strategies to students may be an effective way to improve student commitment 
to long-term academic goals, such as achieving high test scores through test preparation. 

Dignath, C., Buettner, G., & Langfeldt, H. (2008). How can primary school students learn self- 
regulated learning strategies most effectively?: A meta-analysis on self-regulation training 
programmes. Educational Research Review, 3(2), 101-129. 

This is a meta-analysis and discussion of self-regulation training programs in primary school. 
The authors review 48 studies on the effectiveness of self-regulation training programs for 
improving academic achievement, cognitive and metacognitive strategy implementation, and 
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motivation. Results indicated that self-regulation training programs achieve, on average, 
a moderate to large effect size for these outcomes. The largest effect sizes for academic 
achievement were found for mathematics performance. The authors provide a list of 
effective training program characteristics. 

Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and 
passion for long-term goals. Journal of Personality and Social Psychology, 92, 1087-1101. 

This article introduces the construct of “grit," which is defined as perseverance in the pursuit 
of long-term goals. The authors review four studies, which collectively demonstrate how grit is 
positively correlated with the educational attainment of adult workers, retention and achievement 
of West Point military cadets, grade point average (GPA) of Ivy League undergraduates, and 
performance in the National Spelling Bee. The authors argue that grit may be a better indicator 
of achievement than personality traits or IQ, suggesting the relative importance of this 
“non-cognitive trait” to achievement. 

Duckworth, A. L., & Seligman, M. E. P (2005). Self-discipline outdoes IQ in predicting academic 
performance of adolescents. Psychological Science, 16, 939-44. 

This study demonstrates that measures of self-discipline may be better indicators of academic 
performance than tests of cognitive ability. Two separate tests of approximately 300 eighth- 
grade students revealed that combined measures of self-discipline explained twice as much 
variance as tests of cognitive ability alone in predicting later achievement. 

Personality 

Almlund, M., Duckworth, A. L., Heckman, J., & Kautz, T. (2011). Personality psychology and 
economics. In E. A. Hanushek, S. Machin, & L. Wossmann (Eds.), Handbook of the economics 
of education (pp. 1-181). Amsterdam, Netherlands: Elsevier. 

This review of research provides a comprehensive overview of the relationship between 
personality and performance. The authors conclude that of the "Big Five” personality traits (see 
Figure 5 on next page), conscientiousness is the best predictor of educational attainment and 
achievement. The authors demonstrate that measures of conscientiousness can be used to 
predict college grades as well as scores on the SAT; in addition, personality measures are 
predictive of performance on achievement and intelligence tests. The authors present data 
that suggest personality is more malleable than cognitive ability alone, arguing that personality 
holds promise as a potential lever for improving student achievement outcomes. 
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Figure 5. The Five-Factor Model of Personality, or “Big Five” Personality Traits 


Trait 

Characteristics 

Extraversion 

Sociable, gregarious, assertive, talkative, active 

Neuroticism 

Anxious, depressed, angry, embarrassed, emotional, 
worried, insecure 

Agreeableness 

Courteous, flexible, trusting, good-natured, cooperative, 
forgiving, soft-hearted, tolerant 

Conscientiousness 

Careful, thorough, responsible, organized, planful, 
hardworking, achievement-oriented, persevering 

Openness 

Imaginative, cultured, curious, original, broad-minded, 
intelligent, artistically sensitive 


Barrick, M. R., & Mount, M. K. (1991). The big five personality dimensions and job performance: A meta-analysis. 
Personnel Psychology, 44(1), 1-26. 


Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic 
performance. Psychological Bulletin, 135(2), 322-338. 

A meta-analysis of 80 research papers including more than 70,000 total participants, this 
report provides substantial evidence for the predictive relationship between personality 
(specifically conscientiousness) and academic performance. The positive relationship between 
conscientiousness and academic achievement held true even when controlling for intelligence. 

Roberts, B. W., & Del Vecchio, W. F. (2000). The rank-order consistency of personality traits from 
childhood to old age: A quantitative review of longitudinal studies. Psychological Bulletin, 126, 3-25. 

This review utilized a meta-analytic method to determine the average stability of personality 
traits over time. The authors reviewed 152 longitudinal studies and found that personality 
begins to crystallize around age 30, reaching maximum stability between the ages of 50 to 70. 
The authors demonstrate that personality is at its most malleable during the school years and 
that environmental changes during that time account, at least in part, for changes in personality. 

Mindsets 

Bill & Melinda Gates Foundation. (2014). Academic tenacity: Mindsets and skills that promote 
long-term learning. Seattle, WA: Author. 

This report provides a broad overview of the research into non-cognitive factors, with a special 
focus on motivation. The authors present research to demonstrate that interventions and 
policies that promote factors, such as student beliefs, motivation, self-regulation, and school 
culture, can have a large, positive impact on academic outcomes — including grades and test 
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performance — and that results can be sustained overtime. The report lists and summarizes 
various exemplary interventions, applicable across subjects, which are aimed at improving 
academic performance by supporting positive student mindsets, social belonging, self-concept, 
and self-regulation strategies. The effects of the interventions included cutting the racial 
achievement gap in half and increasing student grades by two thirds. The authors conclude 
with guidance on curriculum and staff development that can support practices that foster 
these student attributes and outcomes. 

Walton, G. M., & Cohen, G. L. (2011). A brief social-belonging intervention improves academic 
and health outcomes among minority students. Science, 331, 1447-1451. 

This study demonstrates that a brief (one-hour) intervention with college freshmen, which aims 
to lessen their fears about social belonging in academics, 5 was correlated with improved GPA 
over three consecutive years immediately following the intervention. The authors hypothesized 
that African-American students experience greater fear about social belonging in academics, 
which may impair their overall academic performance. The academic performance gap between 
African-American students who participated in the intervention and Caucasian students was 
reduced by nearly 80 percent in comparison with the same gap for African-American students 
who did not participate in the intervention. 

Cohen, G. L., Garcia, J., Purdie-Vaughns, V., Apfel, N., & Brzustoski, P (2009). Recursive processes 
in self-affirmation: Intervening to close the minority achievement gap. Science, 324, 400-403. 

This report details the two-year follow-up to the Cohen, Garcia, Apfel, and Master (2006) study, 
in which a 15-minute writing exercise, designed to reduce stereotype threat among minority 
students, was correlated with an improvement in academic performance. The authors found 
that the results were sustained two years later, leading to higher GPAs for African-American 
participants, particularly for those who were lower achieving at the start of the study. 
Participants also showed a reduced rate of remediation and grade repetition and 
sustained improvements in self-perception. 

Walton, G. M., & Cohen, G. L. (2007). A question of belonging: Race, social fit, and achievement. 
Journal of Personality and Social Psychology, 92, 82-96. 

In this study, researchers created conditions under which learners would not feel a sense of 
social belonging in academics. The results demonstrated that although Caucasian students 
were largely unaffected by study messaging, African-American students were affected negatively. 
The authors also found that a brief intervention to improve beliefs about social belonging in 
academics eliminated these effects, and African-American students demonstrated improved 
academic performance. 

Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict 
achievement across an adolescent transition: A longitudinal study and an intervention. Child 
Development, 78(1), 246-263. 


5 This means identifying and associating oneself with academically motivated peers. 
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This article details both a longitudinal study and an intervention, both designed to address 
students’ beliefs about the malleability of intelligence. In the longitudinal study, the authors 
found a positive correlation between the belief that intelligence is malleable and student 
performance in mathematics. This was evident from improved mathematics grades over two 
years for students who believed that intelligence can improve with effort, as opposed to those 
who conceptualized intelligence as a fixed trait. In the intervention study, the authors found that 
a brief intervention to teach students about the incremental theory of intelligence (i.e., that 
intelligence/cognitive ability can improve with effort) improved student motivation and reversed 
an existing, downward trend in mathematics grades, relative to students who received an 
alternative intervention focused on improving memory and study skills. 

Cohen, G. L., Garcia, J., Apfel, N., & Master, A. (2006). Reducing the racial achievement gap: 

A social-psychological intervention. Science, 313, 1307-1310. 

This study examines an intervention designed to lead middle school students through a 
brief, 15-minute writing exercise about personal values. The study shows this intervention 
is correlated with a reduction in stereotype threat and improved academic performance 
among minority students. These effects led to reductions in racial achievement gaps in 
GPA, in schools where this was tested, by as much as 40 percent. 

Good, C., Aronson, J., & Inzlicht, M. (2003). Improving adolescents’ standardized test 
performance: An intervention to reduce the effects of stereotype threat. Journal of 
Applied Developmental Psychology, 24, 645-662. 

In this randomized controlled trial, the authors implemented an intervention in which female 
students, minority students, and/or low-income students learned about the growth mindset 
of intelligence. Both students who received the intervention and those who did not worked 
with academic mentors, but only intervention participants learned about the growth mindset 
of intelligence. The authors found that students who received the intervention demonstrated 
improved standardized test scores in reading and mathematics compared with those students 
who did not receive the intervention. 

Motivation 

Cury, F., Elliot, A. J., Da Fonseca, D., & Moller, A. C. (2006). The social-cognitive model of 
achievement motivation and the 2x2 achievement goal framework. Journal of Personality 
and Social Psychology, 90, 666-679. 

This study introduces the 2x2 achievement goal framework and demonstrates how that model 
serves as a predictor of academic performance. The 2x2 model includes two theories or beliefs 
about ability that students may have 6 and two kinds of achievement goals: performance 7 and 


6 The belief that ability is innate and static, or that ability is developed and improves with effort 

7 Receiving high scores 
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mastery. 8 The authors demonstrate that when students perceive themselves as competent, 
they are more likely to adopt beliefs that ability can improve (as opposed to being static) and to 
elect mastery goals as opposed to performance goals. This mindset, in turn, is correlated with 
higher motivation and better academic performance. The authors suggest that this research 
highlights the importance of framing beliefs about ability that encourage a growth mindset. 

Osterman, K. F. (2000). Students’ need for belonging in the school community. Review of 
Educational Research, 70(3), 323-367. 

This article provides a review of the research on the relationship between students’ sense of 
belonging in the school community and student motivation. The author argues that neglecting 
this component of student support and development has a large negative impact on student 
behavior — including motivation — which is correlated with lower academic performance (i.e., GPA), 
more disciplinary issues, and higher dropout rates. The author argues that students should be 
provided with structured opportunities to interact with peers, instruction that encourages 
collaboration, and teacher support and attention to students who struggle with social belonging. 
These strategies are especially critical for secondary students transitioning from elementary and/ 
or middle school, where the drop in social support from teachers and staff is most pronounced. 

Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. 
Contemporary Educational Psychology, 25, 68-81. 

This paper describes the expectancy-value theory of motivation in detail. The authors explore 
changes in children’s and adolescents’ beliefs about ability, their expectations for success, how 
they value academics, and the relationship of those factors to academic performance and their 
choice of activities. The authors highlight several key takeaways as follows: 

1. Student beliefs about their ability and expectations of success are better predictors 
of subsequent grades than previous academic performance. 

2. The degree to which students value a subject is the best predictor of their perseverance 
within that subject. 

3. Positive beliefs about ability, expectations of success, and valuations of activities decline 
across the school years, especially during the middle school transition. 

Elliot, A. J., McGregor, H. A., & Gable, S. (1999). Achievement goals, study strategies, and exam 
performance: A meditational analysis Journal of Educational Psychology, 91(3), 549-563. 

This study evaluates the relationship between achievement goals, learning strategies, and exam 
performance. The authors identify three primary achievement goals: mastery, performance- 
approach, 9 and performance-avoidance. 10 Findings include the following: 


8 Achieving a deep understanding of content 

9 High performance relative to peers 

10 Avoiding perceptions of lower performance relative to peers 
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1. Mastery goals are positively correlated with “deep processing” 11 of content. 

2. Performance-approach goals are positively correlated with surface processing 
(memorization) of content. 

3. Performance-avoidance goals are negatively correlated with “deep processing” of content 
and highly correlated with surface processing. 

4. Performance-approach and mastery goals are associated with higher levels of persistence 
and effort, which are in turn correlated with better exam performance. 

5. Performance-avoidance goals are associated with increased disorganization, which is in turn 
correlated with poor exam performance. 

The authors provide a discussion of the practical educational implications of these 
relationships in their conclusion. 

Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 41(10), 
1040-1048. 

This review synthesizes findings from several decades of research on the relationship between 
motivation and academic achievement. Dweck explains how the type of goals students adopt 
with respect to schoolwork (task performance versus content mastery) affects their patterns 
of achievement. Content mastery goals are associated with: 

1. Increased effort in response to failure. 

2. Higher levels of proficiency. 

3. Higher levels of retention. 

4. Increased challenge-seeking behavior. 

5. Increased transfer of knowledge. 

This effect is found to be present whether or not a child’s ability is high and, in fact, may be 
most pronounced for children who initially test highly on measures of IQ and who are female. 
Dweck concludes with a discussion of implications for practitioners. She argues for a reduced 
emphasis on competitive and individual, task-limited ratings of student performance (i.e. , 
grades on homework and tests), and a greater emphasis on progress-related ratings of student 
performance. She further suggests a reduced emphasis on continuous, positive feedback for 
small tasks (i.e., in-class assignments and homework), which is particularly prevalent with 
low-achieving children, to whom it is most injurious. 

Eccles, J., Midgley, C., & Adler, T. F. (1984). Grade-related changes in the school environment: 
Effects on achievement motivation. In J. G. Nicholls (Ed.), The development of achievement 
motivation (pp. 283-331). Greenwich, CT: JAI Press. 


11 Deep processing involves critically thinking about new information and integrating it with prior knowledge. 
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This book chapter describes changes in students’ achievement motivation, over time, as a 
result of developmental changes and changes in the school environment. The authors argue 
that achievement motivation declines overtime across ages and grades (especially during 
school transition years) and that there are two major school-level variables that influence that 
decline: (1) practices that shift students’ focus to ability assessment and social comparison 
instead of task involvement, and (2) practices that shift pedagogical practice from student- 
centered to teacher-centered control of the learning environment. 


INTERPERSONAL COMPETENCIES: SOCIAL AND EMOTIONAL SKILLS 

Civic Enterprises., Bridgeland, J., Bruce, M., & Hariharan, A. (2013). The missing piece: A national 
teacher survey on how S EL can empower children and transform schools. Collaborative for 
Academic, Social, and Emotional Learning. Chicago, IL: Civic Enterprises. 

This report provides a broad overview of the role of social and emotional learning (SEL) in 
education. The authors draw on research on theory and best practice, as well as teacher and 
student survey reports, to provide an overview of SEL research and the practitioner landscape. 
Their definition of SEL highlights the interpersonal dimensions of teamwork, cooperative learning, 
and social skills, as well as self-awareness and self-management (see Figure 6). The authors 
provide evidence of the benefits of SEL — including higher student engagement, intrapersonal 
problem-solving skills, and decreased truancy, drug abuse, bullying, and violence — and best- 
practice models to promote SEL. They conclude by arguing for the formal integration of SEL goals 
and skills into school curricula, teacher training, accountability measures, state standards, 
funding mechanisms, and the broader education research agenda. 

Figure 6. SEL Core Competencies 


Social and Emotional Learning Core Competencies 

CASEL has identified five interrelated sets of cognitive, affective and 
behavioral competencies. The definitions of the five competency 
clusters for students are: 

• Self-awareness: The ability to accurately recognize one's 
emotions and thoughts and their influence on behavior. This 
includes accurately assessing one's strengths and limitations and 
possessing a well-grounded sense of confidence and optimism. 

• Self-management: The ability to regulate one's emotions, 
thoughts, and behaviors effectively in different situations. This 
includes managing stress, controlling impulses, motivating 
oneself, and setting and wording toward achieving personal and 
academic goals. 

• Social awareness: The ability to take the perspective of and 
empathize with others from diverse backgrounds and cultures, to 
understand social and ethical norms for behavior, and to 
recognize family, school, and community resources and supports. 

• Relationship skills: The ability to establish and maintain healthy 
and rewarding relationships with diverse individuals and groups. 

This includes communicating clearly, listening actively, 
cooperating, resisting inappropriate social pressure, negotiating conflict constructively, and seeking and offenng help when needed. 

• Responsible decision making: The ability to make constructive and respectful choices about personal behavior and social interactions 
based on consideration of ethical standards, safety concerns, social norms, the realistic evaluation of consequences of various actions, and 
the well-being of self and others. 
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Duncan, G., & Magnuson, K. (2011). The nature and impact of early achievement skills, attention 
skills, and behavior problems. In G. J. Duncan & R. J. Murnane (Eds.), Whither opportunity: 
Rising inequality, schools, and children’s life chances (pp. 47-70). New York, NY: Russell 
Sage Foundation. 

This book chapter provides an overview of the relationships between achievement, attention, 
behavior problems, and mental health in education. The authors find that children from low 
socioeconomic backgrounds have significantly lower school-entry academic skills and social 
and emotional skills than their peers from more affluent backgrounds. The lower academic 
skills persist across elementary school, while the social and emotional skills gap nearly 
doubles during that period. The authors find that persistent deficits in childhood are strong 
predictors of lower achievement later on, but that single point-in-time indicators are relatively 
weak predictors. For behavior problems, early onset is an indicator for persistent problems, 
whereas adolescent onset rarely persists beyond the adolescent period. Overall, school-entry 
achievement predictors are strongest when measured after age 10 and over time for persistence. 

Durlak, J. A., Dymnicki, A. B., Taylor, R. D., Weissberg, R. P, & Schellinger, K. B. (2011). The impact 
of enhancing students’ SEL: A meta-analysis of school-based universal interventions. Child 
Development, 82(1), 405-432. 

This meta-analysis of 213 school-based universal SEL programs includes 270,034 
kindergarten-through-high school students. The authors found that SEL program participants 
demonstrated significantly better social and emotional skills, attitudes, behavior, and academic 
performance compared with students who did not have access to SEL programming. This 
finding was evident from improved GPA and standardized test scores (an average 11 percent 
increase), fewer disciplinary incidents, reduced levels of psychopathology, and more positive 
prosocial behaviors between students and their peers, parents, and teachers. These effects 
were still evident at a one-year follow-up. The authors cite the use of the SAFE (sequenced, 
active, focused, and explicit) framework being a significant driver of programs’ positive results. 

Durlak, J. A., Weissberg, R. P, & Pachan, M. (2010). A meta-analysis of after-school programs that 
seek to promote personal and social skills in children and adolescents. American Journal of 
Community Psychology, 45, 294-309. 

This review of afterschool programs uses a meta-analysis to evaluate the impact of afterschool 
programs on social and emotional skills as well as academic achievement. The authors introduce 
SAFE programs (see Figure 7 on next page) as a model for effective afterschool programs. They 
found that when programs adopted this model, students demonstrated improved emotional 
well-being, more positive social behaviors, fewer problem behaviors, and significant improvement 
in achievement test scores and grades relative to students who did not receive the program 
or who received a non-SAFE program model. 
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Figure 7. SAFE Program Model 


Feature 

Definition 

Sequenced 

The program uses a connected and coordinated set 
of activities to achieve their objectives relative to 
skill development. 

Active 

The program uses active forms of learning 
to help youth learn new skills. 

Focused 

The program has at least one component devoted 
to developing personal or social skills. 

Explicit 

The program targets specific personal 
or social skills. 


Duncan, G., Dowsett, C., Classens, A., Magnuson, K., Huston, A., Klebanov, P, Pagani, L., et al. 
(2007). School readiness and later achievement. Developmental Psychology, 43(6), 1428-1446. 

This article presents the results of six longitudinal studies designed to demonstrate the 
relationship between three key foundations for school readiness — academic skills, attention 
skills, and social and emotional skills — and academic achievement in reading and mathematics 
in third and fifth grades. The authors found that the three strongest predictors were school- 
entry mathematics, reading, and attention skills (in that order) and that measures of social and 
emotional skills were not significantly predictive factors, even for children with high levels of 
problem behaviors. These results were consistent across gender and socioeconomic status. 

Vitaro, F., Brendengen, M., Larose, S., & Tremblay, R. E. (2005). Kindergarten disruptive behaviors, 
protective factors, and educational achievement by early adulthood. Journal of Educational 
Psychology, 97, 617-629. 

This study includes a sample of 4,330 children in its analysis of the relationship between 
hyperactivity-inattention and aggressiveness-opposition and noncompletion of high school by 
early adulthood. The authors found that hyperactivity-inattention was a stronger predictor for 
noncompletion than was aggressiveness-opposition, and that this relationship was moderated 
by prosocial behaviors and parenting style. The authors suggest that certain student behaviors 
(inattention-hyperactivity) need more critical attention and intervention early on to avert later 
deficits in performance. 

Greenberg, M. I, Weissberg, R. P, O’Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., & Elias, 

M. J. (2003). Enhancing school-based prevention and youth development through coordinated 
social, emotional, and academic learning. American Psychologist, 58(6-7), 466. 

This article provides a broad overview of prevention and youth development programs, using the 
SEAL (social, emotional, and academic learning) model. The author reviews evidence from the 
field supporting the use of these programs to improve student outcomes for interpersonal 
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skills, mental health, and academic performance and behavior. The authors found the most 
effective programs are those that begin early and are reinforced through aligned programming in 
later grades. Effective programs also are fully integrated into the whole school, for all students. 
This programming requires professional development for teachers, administrators, and staff as 
well as the active participation of parents and students. Successful implementation depends 
on using research-based practices and must be coupled with data collection to monitor and 
maintain effectiveness. 


Impact on Career Readiness 

The following citations include those studies that seek to define the role and impact that lifelong 
learning skills have on career readiness and the labor market. The majority of this research has 
been conducted by economists, in conjunction with leaders in psychology, education, and other 
social science fields. 

Almlund, M., Duckworth, A., Heckman, J., & Kautz, T. (2011). Personality psychology and 

economics. In E. A. Hanushek, S. Machin, & L. Wossmann (Eds.), Handbook of the economics 
of education (pp. 1-181). Amsterdam, Netherlands: Elsevier. 

This research review provides a comprehensive overview of the relationship between 
personality and performance. The authors conclude that of the "Big Five” personality traits, 
conscientiousness is the best predictor of career success. Conscientiousness is a strong 
predictor of overall job performance and wages across a wide variety of occupations. They argue 
that because personality is relatively stable over the life course, it can be used as a reliable 
predictive measure. In addition, because it also is more malleable than cognitive ability, it holds 
promise as a lever on which to act to improve the achievement outcomes (i.e. , GPA, standardized 
test scores, cognitive skills tests, and postsecondary attainment) with which it is associated. 

Brunello, G., & Schlotter, M. (2010). Non-cognitive skills and personality traits: Labour market 
relevance and their development in education and training systems (IZA Discussion Paper 
No. 5743). Analytical Report for the European Commission prepared by the European 
Expert Network on Economics of Education. 

In this article, the authors find that inter- and intrapersonal skills play a central role in determining 
educational and career achievement outcomes, such as dropout rates, postsecondary enrollment, 
employment, and earnings. They argue that the results of cognitive tests — which also predict 
educational and career achievement — are likely influenced in part by inter- and intrapersonal skills 
that improve performance on those same tests. The authors argue that because inter- and 
intrapersonal skills may be more malleable than cognitive skills, they may prove more useful for 
intervention efforts. 

Cunha, F., Heckman, J. J., & Schennach, S. M. (2010). Estimating the technology of cognitive and 
noncognitive skill formation (IZA Discussion Paper No. 4702). Econometrica, 78(3), 883-931. 

In this paper, the authors demonstrate that inter- and intrapersonal skills promote the 
development of cognitive skills, but not vice versa. They find that early investment in cognitive 
skills yields the highest returns on investment — in terms of improved academic performance — 
but that those returns diminish over time. However, investments in inter- and intrapersonal 
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skills can be effective and retain their impact on both academic performance and social 
success, even at later ages. This finding suggests that cognitive skills promotion should 
be a focus in the early years, but that a focus on inter- and intrapersonal skills may be 
more effective later (e.g., for remediation efforts) in adolescence. 

Borghans, L., ter Weel, B., & Weinberg, B. A. (2005). People people: Social capital and the 
labor-market outcomes of underrepresented groups (IZA Discussion Paper No. 1494). 

This large-scale analysis of non-cognitive skills and career achievement data from the United 
States, Britain, and Germany demonstrates the value of interpersonal skills in the modern 
workplace. The authors find that higher levels of interpersonal skill are associated with 
attainment and sorting into (or seeking out) jobs in which people earn higher wages. The 
authors illustrate how computerization and organizational changes in the modern workplace 
demand higher interpersonal skills. Furthermore, the authors find that occupations that require 
higher levels of interpersonal skills favor women relative to men, and favor minorities least. 

Borghans, L., ter Weel, B., & Weinberg, B. (2007). Interpersonal styles and labor market outcomes 
(NBER Working Paper No. 12846). Cambridge, MA: National Bureau of Economic Research. 

This paper provides a framework for understanding how different interpersonal styles influence 
how individuals select career pathways and subsequent career achievement. The authors make 
a distinction between two major components of interpersonal style: caring and directness. They 
find that, overall, directness (clear communication) has a higher value than caring (cooperation) 
and that behavioral measures of youth sociability are predictive of the jobs they will hold as adults. 

Carneiro, P, Crawford, C., & Goodman, A. (2007). The impact of early cognitive and non-cognitive 
skills on later outcomes. London, UK: Centre for the Economics of Education, London School 
of Economics. 

In this article, the researchers use data from a large longitudinal study, finding that non-cognitive 
skills, even at early ages, are strong predictors for academic and career achievement, specifically, 
dropout, postsecondary attainment, employment, and wages. The authors suggest that inter- and 
intrapersonal skills may be more malleable than cognitive skills, and therefore may be a more 
effective and efficient focus for education and social policy efforts to improve college and career 
readiness and success. 

Heckman, J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities 
on labor market outcomes and social behavior. Journal of Labor Economics 24(3), 411-482. 

In this paper, the authors determine that non-cognitive skills have an important impact on 
educational and career achievement outcomes. The authors find that learners who receive a 
GED have similar cognitive skills to their peers who receive a high school diploma, but show 
differences in wages that may be attributable to variations in non-cognitive skills. They find that 
this same difference in non-cognitive skills may explain why some adolescents engage in high- 
risk behaviors and may contribute to their decisions of whether or not to pursue postsecondary 
education. The authors suggest that the lasting benefits of early childhood interventions are 
largely attributable to improvements in non-cognitive ability and show that the effects of those 
interventions are greater for females than they are for males. 
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Measures 


This section highlights reports on measures for the assessment of lifelong learning skills. Due to 
the developing nature of the construct and its many components, there are relatively few validated 
and comprehensive assessments of lifelong learning skills. Most assessments represent an 
attempt to capture only a subset of lifelong learning skills, and have not yet been specifically 
validated as predictors of college and career readiness or success. 


MISSION SKILLS ASSESSMENT (MSA) 

Independent Schools Data Exchange (INDEX). (2014). MSA— A tool to alter the way schools think 
about education: 2014 NAIS conference [PowerPoint slides]. Retrieved from http://indexgroups. 
org/msa/docs/2014%20MSA%20NAIS%20Presentation.pdf 

This presentation contains an overview of non-cognitive skills, the development of the MSA, 
and initial findings of validity and reliability of the test. The test is Web-based and seeks to 
assess sixth to eighth graders on six non-cognitive factors: creativity, resilience, curiosity, 
teamwork, ethics, and time management. It includes student self-assessment and biographical 
responses, teacher ratings, and situational judgment tests. The assessment is currently being 
piloted among 13,000 students at 70 schools and has been previously tested among more 
than 4,000 students across 38 schools. The authors show that the MSA is reliable and adds 
predictive value for academic performance (i.e., GPA and scores on standardized tests) when 
paired with a cognitive skills test. 

Soland, J., Hamilton, L. S., & Stecher, B. M. (2013). Measuring 21st century competencies: 
Guidance for educators. Global Cities Education Network. 

In this report on assessments of lifelong learning skills, the authors highlight the MSA as an 
exemplary assessment for its measurement of multiple skills using several sources (i.e., a 
student survey, multiple-choice questions, and teacher observation). The authors recommend 
using the MSA as a schoolwide assessment that can promote the teaching of lifelong learning 
skills while also being low cost and easy to understand. 


TAUCK FAMILY FOUNDATION FORMATIVE ASSESSMENT TOOL 

Child Trends. (2014). Measuring elementary school students’ social and emotional skills: Providing 
educators with tools to measure and monitor social and emotional skills that lead to academic 
success. Bethesda, MD: Author. 

This report details the development of a new tool designed to measure social and emotional 
skills in elementary school students (Grades K-5). The authors provide an overview of how 
social and emotional skills influence student academic (e.g., course grades) and social 
achievement. They propose a five-factor model consisting of the social and emotional skills 
that are most influential for student academic and social achievement outcomes, including 
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self-control, persistence, mastery orientation, academic self-efficacy, and social competence. 
The authors provide a definition for each skill and review the literature linking it to academic 
achievement. Next, they provide a detailed description of the development of their assessment, 
which draws directly from survey questions used in well-established measures of each skill 
within the five-factor model. They also include a review of each of those individual measures, 
along with a chart describing them and citations for further review. The pilot study of their 
assessment, in conjunction with Bridgeport (Connecticut) Public Schools and Yale University, 
demonstrated that the measure is valid and reliable. The surveys are presented in the 
appendixes along with the survey’s evaluation results. A suggested implementation program 
is described for those interested in using the new surveys in their school(s) or district(s). 


SMARTER BALANCED AND PARTNERSHIP FOR ASSESSMENT 
OF READINESS FOR COLLEGE AND CAREERS (PARCC) 

ASSESSMENT CONSORTIA 

Herman, J. L., & Linn, R. L. (2013). On the road to assessing deeper learning: The status of 
Smarter Balanced and PARCC assessment consortia (CRESST Report 823). Los Angeles, CA: 
University of California, National Center for Research on Evaluation, Standards, and Student 
Testing (CRESST). 

This report provides a comprehensive background on and analysis of the tests being developed 
by the Smarter Balanced and PARCC assessment consortia. The authors find that the tests 
demonstrate the potential to effectively measure mastery of academic content, while also 
measuring cognitive strategies for complex thinking, communication, and problem solving. 
The authors caution that political, financial, and technical limitations may degrade the reliability 
and validity of the tests. Budget limitations in some states may lead some to only partially 
participate (the cost is estimated to be $20 per student), and although the use of technology 
to conduct the assessments is desirable, ensuring that all schools have the requisite hardware 
may be difficult. These challenges could threaten the successful implementation and usability 
of data gathered from these tests. 
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PROGRAM FOR INTERNATIONAL STUDENT ASSESSMENT (PISA) 


OECD. (2013), PISA 2012 assessment and analytical framework: Mathematics, reading, science, 
problem solving and financial literacy. OECD Publishing. 

This report contains an overview of the guiding framework for the PISA 2012 exam. 
Particularly relevant to the assessment of lifelong learning skills is chapter four, which 
reviews the measurement framework for problem solving and breaks down the components 
of each question within that section. PISA 2012 questions vary by problem context, 12 the nature 
of the problem, 13 and the cognitive components of problem solving that are measured. 14 Each 
question reveals student performance on every component in the process of problem solving 
and provides students with multiple opportunities to demonstrate their problem-solving ability. 
This type of construction and assessment of each question allows for more intricate and 
domain-general measurement of problem-solving skills. 

OECD. (2013), PISA 2015 draft collaborative problem solving framework. OECD Publishing. 

This document contains a draft version of the measurement framework in development for 
the 2015 PISA exam, describing the assessment components integrated into each question. 
The collaborative problem-solving framework is particularly relevant to “21st century skills” 
development, as it simultaneously demands interpersonal and cognitive skills from students 
as they solve each problem. The test presents students with complex problems that can only 
be solved through collaboration with a virtual peer. The entire collaborative process, as well 
as the cognitive process and the correctness of the student’s answer, are taken into account 
to determine the final score. 

Greif, S., Holt, D. V., & Funke, J. (2013). Perspectives on problem solving in educational 
assessment: Analytical, interactive, and collaborative problem solving. Journal of Problem 
Solving, 5(2), 71-91. 

The authors of this study are the developers of the problem-solving portion of PISA. They use 
the problem-solving sections from the 2003, 2013, and 2015 PISA to compare and contrast 
three kinds of problem solving: analytical, interactive, and collaborative. Analytical problem 
solving is that in which all of the necessary information is presented in the problem, interactive 
problem solving requires additional information to be abstracted or input to solve the problem, 
and collaborative problem solving requires interaction with another person to solve the problem. 
The authors discuss the advantages and limitations associated with the assessment of each 
kind of problem solving as well as new directions for future research. Analytic problem solving 
limits the student by not allowing for exploration and testing of the problem context. Interactive 
problem solving attempts to correct for this through computer simulation, but is inherently 
limited compared with real-world interaction. The collaborative problem-solving component 
of the forthcoming 2015 PISA addresses the limitations of interactive problem solving by 

12 Questions related to personal, technological, or social problems or scenarios 

13 Static, or requiring no additional information, and interactive, or requiring abstraction of facts 

14 Exploring and understanding, representing and formulating, planning and executing, and monitoring and reflecting 
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introducing live or simulated collaborators with whom the test-taker must work to solve the 
problem. Additional research is necessary, however, to determine how best to simulate the real- 
world scenario of collaborative problem solving, while still maintaining standardization of the test. 


ASSESSMENTS FOR SOCIAL AND EMOTIONAL LEARNING 

Haggerty, K., Elgin, J., & Woolley, A. (2011). Social-emotional learning assessment measures 
for middle school youth. Social Development Research Group. University of Washington. 
Commissioned by the Raikes Foundation. 

This guide to SEL assessments is designed to help educators choose SEL assessments that 
match their needs. The authors reviewed 73 assessments of SEL and found 10 that met their 
criteria for effectiveness and efficiency, using CASEL’s (Collaborative for Academic, Social, and 
Emotional Learning) framework for SEL as a guide. The assessments are reviewed individually 
for technical properties, such as validity; format, such as survey or multiple-choice; age range; 
and usability concerns, such as cost and training requirements. Many of these assessments 
are available for use with younger as well as older children and adolescents. The assessments 
are designed to measure both inter- and intrapersonal skills, with some also evaluating general 
mental health. The authors also provide this information in a condensed chart along with a 
glossary of terms and sample versions of the tests. 


COLLEGE- AND CAREER-READY SCHOOL DIAGNOSTIC 

Conley, D. T, McGaughy, C., Kirtner, J., Van Der Valk, A., & Martinez-Wenzl, M. T. (2010, April). 
College readiness practices at 38 high schools and the development of the CollegeCareerReady 
School Diagnostic tool. Paper presented at the 2010 annual meeting of the American 
Educational Research Association, Denver, CO. 

This report details the investigation of college and career readiness practices in high schools 
and the subsequent development of the CollegeCareerReady School Diagnostic. The assessment 
includes a Web-based application that assesses, tracks, and provides feedback based on college 
and career readiness practice surveys administered to teachers, administrators, counselors, and 
students. The surveys are designed to reveal the degree to which schools are following the college 
and career readiness framework, developed by Conley. The framework focuses on four dimensions: 
cognitive strategies, content knowledge, transition knowledge and skills, and learning skills and 
techniques. Initial results indicated that schools used findings from the survey to inform decisions 
about future steps to improve practice promoting college- and career-ready skills. Additional 
research is needed to determine what student outcomes were affected by use of the survey. 

Lombardi, A. R., Conley, D. T, Seburn, M. A., & Downs, A. M. (2013). College and career readiness 
assessment: Validation of the Key Cognitive Strategies framework. Assessment for Effective 
Intervention, 38(3), 163-171. 
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This study investigates the validity of the CollegeCareerReady School Diagnostic, with respect 
to the Key Cognitive Strategies 15 (KCS) portion of the diagnostic for students. This portion of 
the diagnostic involves a self-report measure that probes student use of the KCS. The authors 
found that the measure was valid and suggest pairing it with the College-Readiness Performance 
Assessment System (C-PAS) to determine the extent to which schools are successful in teaching 
skills for deeper learning. 


COLLEGE-READINESS PERFORMANCE ASSESSMENT SYSTEM 

Conley, D. (2009, April). Formative assessment for college readiness: Measuring skill and growth 
in five key cognitive strategies associated with postsecondary success. Paper presented at the 
2009 annual meeting of the American Educational Research Association, San Diego, CA. 

This report describes the initial validation of an assessment designed to capture acquisition 
of college readiness cognitive thinking skills in mathematics and English language arts (ELA). 
The system comprises a set of tasks in mathematics and ELA, which the teacher assigns to 
students, once in the fall and once in the spring. The tasks are then graded according to the 
students’ use of five cognitive strategies within the KCS framework (problem formulation, 
research, interpretation, communication, precision/accuracy). The author argues that this 
assessment effectively measured the acquisition of these cognitive skills. 

Baldwin, M., Seburn, M., & Conley, D. T. (2011, April). External validity of the Coliege-readiness 
Performance Assessment System (C-PAS). Paper presented at the 2011 annual meeting of 
the American Educational Research Association, New Orleans, LA. 

This study found that the C-PAS was positively correlated, to a modest-to-moderate degree, 
with test scores in ELA and mathematics on the New York Regents exam — a measure of 
college eligibility administered in Grades 9-12. The authors argue that these findings indicate 
C-PAS can be used as a predictor for achievement and college readiness, which in turn can 
inform data-driven instructional practices to improve student performance. 


MICRODYN APPROACH 

Greiff, S., & Funke, J. (2009). Measuring complex problem solving: The MicroDYN approach. 

In F. Scheuermann & J. Bjornsson (Eds.), The transition to computer-based assessment: New 
approaches to skills assessment and implications for large-scale testing. Luxembourg: Office 
for Official Publications of the European Communities. 

This report describes the development and empirical testing of MicroDYN — a test for complex 
problem solving. The authors explain how the test is able to assess the skill of complex 
problem solving on a general, rather than content-specific, level. The test is computer based 
and allows for analysis of the students’ problem-solving process, in addition to their answer. 


15 These strategies include problem formulation, research, interpretation, communication, and precision/accuracy. 
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Students are asked to complete short, five-minute exercises in which they must draw causal 
inferences from complex systems. For each question, the student is given an exploration phase 
(time to manipulate the model), a modeling phase (time to develop the inferred relationships), 
and a control phase (time to provide an answer). The authors argue that the MicroDYN approach 
will allow for more accurate testing of problem solving, at a content-general level, by using 
multiple, brief test items and intentionally assessing well-established components of problem 
solving derived from the cognitive psychology literature. 


GRIT SCALE 

Duckworth, A. L., & Quinn, P D. (2009). Development and validation of the short grit scale (Grit-S). 
Journal of Personality Assessment, 91, 166-174. 

This article introduces the concept of “grit” 16 and its measurement utility for predicting 
various achievement outcomes, including academic success. The Grit Scale uses a brief, 
eight-question self-report assessment to measure student effort and interest. The Grit Scale 
has been empirically tested with adolescents in school, adults at work, West Point military 
cadets, and spelling bee competitors. Grit is a predictor of multiple achievement outcomes, 
including GPA, educational attainment and career dedication, retention of military cadets, 
and performance in competition. 


RESILIENCE AND YOUTH DEVELOPMENT MODULE 
OF THE HEALTHY KIDS SURVEY 

Hanson, T. L., & Kim, J. 0. (2007). Measuring resilience and youth development: The psychometric 
properties of the Healthy Kids Survey (Issues & Answers Report, REL 2007-No. 034). Washington, 
DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Evaluation and Regional Assistance, Regional Educational Laboratory West. 

This report contains a detailed overview and analysis of the Resilience and Youth Development 
Module of the Healthy Kids Survey, which is administered to roughly 600,000 students annually 
in California. The authors point out the various non-cognitive skills constructs that the self-report 
measure evaluates, including self-efficacy, empathy, problem solving, self-awareness, school 
support, and home support — all broadly related to resilience. After reviewing the reliability and 
validity of the measure, they find that although it may be useful for district-level snapshots of 
student resilience, it should not be used to compare individual student differences or as a 
longitudinal measure. They suggest modifications and improvements for the measure, such 
as removing or adding questions, and provide external resources and other related 
assessments for review. 


16 Grit is defined as perseverance in the pursuit of long-term goals. 
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ACADEMIC MOTIVATION SCALE (AMS) 


Grouzet, F. M., Otis, N., & Pelletier, L. G. (2006). Longitudinal cross-gender factorial invariance 
of the Academic Motivation Scale. Structural Equation Modeling, 13(1), 73-98. 

This study provides an analysis of the reliability and validity (see definitions below) of the 
AMS. The AMS uses a self-report method to probe for student motivation and regulation 
skills. The study included a three-year longitudinal model involving more than 600 participants. 
The authors find that the scale is consistent over time. The authors also cite previous research 
on the scale, which has provided strong evidence for its predictive power for final course grades 
and dropout risk. 


Validity refers to the degree to which a measure can be said to be accurately measuring 
that thing which it purports to measure, instead of being influenced by or capturing 
something else. 

Reliability refers to the extent to which the results from a test can be reproduced in a 
consistent manner, over time, when measuring the same thing (e.g., within the same 
person or groups of people, and across similar people or similar groups). 

Fortier, M. S., Vallerand, R. J., & Guay, F. (1995). Academic motivation and school performance: 
Toward a structural model. Contemporary Educational Psychology, 20(3), 257-274. 

This study confirms the predictive power of the AMS. The authors find that student scores 
account for 28 percent of the variance in final course grades. 


MOTIVATED STRATEGIES FOR LEARNING QUESTIONNAIRE (MSLQ) 


Pintrich, R R., & De Groot, E. (1990). Motivational and self-regulated learning components 
of classroom academic performance. Journal of Educational Psychology, 82, 33-40. 

This is the first empirical study of the MSLQ, tested among 173 seventh-grade students. The 
MSLQ is a student self-report measure which includes 56 items probing student motivation, 
cognitive strategy use, metacognitive strategy use, and management of effort. The authors 
found that this measure is an accurate predictor of academic performance, as measured 
by scores on in-class assignments, exams and quizzes, essays, and semester grades. 
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Pintrich, P R., Smith, D. A., Garcia, I, & McKeachie, W. J. (1993). Reliability arid predictive validity 
of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological 
Measurement, 53(3), 801-813. 

This report provides an analysis of the reliability and predictive validity of the MSLQ, as tested 
with college students. The authors find that reliability is strong and that the measure is 
predictive of academic performance as reflected by final course grades across content areas. 


SELF-REGULATED LEARNING INTERVIEW SCHEDULE 


Zimmerman, B. J., & Martinez-Pons, M. (1986). Development of a structured interview for 
assessing student use of self-regulated learning strategies. American Educational Research 
Journal, 23, 614-628. 

This study describes the development and initial testing of the Self-Regulated Learning 
Interview Schedule with 80 lOth-grade students. The assessment is conducted by interview 
and asks students about their use of 14 self-regulated learning strategies. The authors found 
that the measure was predictive of performance on a standardized test of academic achievement. 
It was found to be a better predictor of said test score than socioeconomic status or gender. 

Zimmerman, B. J., & Martinez-Pons, M. (1988). Construct validation of a strategy model of student 
self-regulated learning. Journal of Educational Psychology, 80(3), 284-290. 

This article contains an analysis of the validity of the Self-Regulated Learning Interview Schedule. 
Findings indicate that the assessment measures student use of self-regulated learning strategies 
and that results are not influenced by other student characteristics, such as student verbal 
expressiveness or student achievement on standardized tests. The assessment does correlate 
highly with teacher ratings of students’ self-regulated learning strategy use, suggesting that the 
assessment is a valid measure. 
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COLLEGE & CAREER 
READINESS & SUCCESS Center 

at American Institutes for Research ■ 

1000 Thomas Jefferson Street NW 
Washington, DC 20007-3835 
800.634.0503 

www.ccrscenter.org 



AIK 


American institutes for Research ' 


www.air.org 


This resource is offered by the College and Career Readiness and Success Center (CCRS Center), a central source of 
information and expertise on postsecondary success issues that does not endorse any interventions or conduct field 
studies. Funded by the U.S. Department of Education, the CCRS Center serves regional comprehensive centers in their 
work to build the capacity of states across the nation to effectively implement the goals of the Elementary and Secondary 
Education Act related to college and career readiness. The CCRS Center is housed at American Institutes for Research 
(AIR) and partners with other leading education organizations such as the American Youth Policy Forum and Quill 
Research Associates, LLC. 
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